INVARIANT SUBSPACES OF THE QUASINILPOTENT DT-OPERATOR 



KEN DYKEMA AND UFFE HAAGERUP 

Abstract. In 0] we introduced the class of DT-operators, which are modeled by certain 
upper triangular random matrices, and showed that if the spectrum of a DT-operator is 
not reduced to a single point, then it has a nontrivial, closed, hyperinvariant subspace. In 
this paper, we prove that also every DT-operator whose spectrum is concentrated on a 
single point has a nontrivial, closed, hyperinvariant subspace. In fact, each such operator 
has a one-parameter family of them. It follows that every DT-operator generates the von 
Neumann algebra L(F 2 ) of the free group on two generators. 



1. Introduction 

Let "K be a separable, infinite dimensional Hilbert space and let r B{ { K) be the algebra of 
bounded operators on "K. Let A 6 23 (!K). An invariant subspace of A is a subspace JCo C "K 
such that A("K ) C "K , and a hyperinvariant subspace of A is a subspace "Ho of "K that is 
invariant for every operator B G B(IK) that commutes with A. A subspace of "K is said to be 
nontrivial if it is neither {0} nor "K itself. The famous invariant subspace problem for Hilbert 
space asks whether every operator in has a closed, nontrivial, invariant subspace, and 

the hyperinvariant subspace problem asks whether every operator in !B(!K) that is not a scalar 
multiple of the identity operator has a closed, nontrivial, hyperinvariant subspace. 

On the other hand, if M. C !B(IK) is a von Neumann algebra, a closed subspace Jio of 
3~C is affiliated to M. if the projection p from J{ onto "Kq belongs to M.. It is not difficult 
to show that every closed, hyperinvariant subspace of A is affiliated to the von Neumann 
algebra, VF*(A), generated by A. The question of whether every element of a von Neumann 
algebra Ai has a nontrivial invariant subspace affiliated to M. is called the invariant subspace 
problem relative to the von Neumann algebra A4. 

In we began using upper triangular random matrices to study invariant subspaces 
for certain operators arising in free probability theory, including Voiculescu's circular op- 
erator. In the sequel jl], we introduced the DT-operators; these form a class of operators 
including all those studied in [3] . (We note that the DT-operators were defined in terms of 
approximation by upper triangular random matrices, and have been shown in jH] to solve a 
maxmimization problem for free entropy.) We showed that DT-operators are decomposable 
in the sense of Foia§, which entails that those DT-operators whose spectra contain more 
than one point have nontrivial, closed, hyperinvariant subspaces. In this paper, we show 
that also DT-operators whose spectra are singletons have (a continuum of) closed, nontriv- 
ial, hyperinvariant subspaces. These operators are all scalar translates of scalar multiples of 
a single operator, the DT(5 , l)-operator, which we will denote by T. 
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The free group factor L(F 2 ) C !B(CK) is generated by a semicircular element X and a 
free copy of L°°[0, 1], embedded via a normal *-homomorphism A : L°°[0, 1] — > L(F 2 ) such 
that r o A(/) = f Q f(t)dt, where r is the tracial state on L(F 2 ). Thus X and the image 
of A are free with respect to r and together they generate L(F 2 ). As proved in |U §4], the 
DT(<5 , l)-operator T can be obtained by using projections from A(L°°[0, 1]) to cut out the 
"upper triangular part" of X; in the notation of jjj §4], T = 1XT(X, A). It is clear from this 
construction that each of the subspaces "Kt = A(l[ 0! t])5f is an invariant subspace of T. We 
will show that each of these subspaces is affiliated to W*(T) by proving Do e W*(T), where 
Dq = A(id[o,i]) and id[o,i] is the identity function from [0, 1] to itself. Since X = T + T*, this 
will also imply W*(T) = L(F2). We will then show that each "K t is actually a hyperinvariant 
subspace of T, by characterizing K t as the set of vectors ^eM such that ||T fc £|| has a certain 
asymptotic property as k — > oo. 

2. Preliminaries and statement of results 



In jH §8], we showed that the distribution of T*T is the probability measure fi on [0,e] 
given by 

dfj,(x) = (p(x)dx 
where <p: (0, e) — > R + is the function given uniquely by 



sm v 



exp(t>cotf ) I = — sinv exp(— vcotv), < v < it. 



71 



Proposition 2.1. Let F(x) = J Q X ip(t)dt, x e [0, e]. Then 



smv 



exp(f cotv) 



v 1 sin v 
1-- + : 

7T 7T V 



< V < 7T. 



Proof. From the proof of |H Thm. 8.9] we have that 

sin v 



o : v 



exp(v cot v) 



is a decreasing bijection from (0, ir) onto (0,e). Hence 

(p(t)dt = - 



F(a(v)) 



-[p{a{u))a{u)\l + 



ip(a(u))a'(u)du 
d 



1 

7T 



sin 2 m 



■it 



+ 



7T 



du 

u sinw 



V?(cr(n)) ) a(u)du 

1 sin 2 f 



+ 1- 



□ 



(2.1) 



(2.2) 



(2.3) 



sinu u it v it 

The following is the central result of this paper. 
Theorem 2.2. Let S k = k((T k )*T k )^ , k = 1, 2, . . . . T/ien a(S k ) = [0, e] /or all k E N and 

lim \\F(S k ) - D0II2 = for k -> oo. 



In particular D e W*(T). Therefore "K t = l[ 0)t ](Do)^ = A(l[ , t ])IK, < t < 1 is a one- 
parameter family of nontrivial, closed, T -invariant subspaces affiliated with W*(T). 

Corollary 2.3. W*(T) = L(F 2 ). Moreover, i/Z is any DT-operator, then W*(Z) = L(F 2 ). 
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Proof. As described in the introduction, with T = U7(X, A) G W*(XUX(L°°[0, 1])) = L(F 2 ), 
from Theorem I2~21 we have D G W*{T). Since clearly X G W*(T), we have W*(T) = L(F 2 ). 
By jH Thm. 4.4], Z can be realized as Z = D + cT for some D G A(L°°[0, 1]) and c > 0. 
By jU Lem. 6.2], T G W*(Z), so W*(Z) = L(F 2 ). □ 

We now outline the proof of Theorem 12.21 Let M be a factor of type Hi with tracial 
state tr, and let A,B<E M sa . By [TJ §1], there is a unique probability measure \ia,b on 
cr(yl) x cr(-B), such that for all bounded Borel functions /, g on a(A) and o~(B), respectively, 
one has 

tr(f(A)g(B)) = |y f(x)g{y)d» A)B {x). (2.4) 

ct(A)xct(B) 

The following lemma is a simple consequence of ()2.4j) (cf. ^1 Proposition 1.1]). 

Lemma 2.4. Lei A, B and [ia,b be as above, then for all bounded Borel functions f and g 
on cr(A) and o~(B), respectively, 

\\f(A)-g(B)\\l= jj \f(x)-g(y)\ 2 dv A A^y)- (2-5) 

cr(A)xcr(B) 

We shall need the following key result of Sniady [7j. Strictly speaking, the results of jTj 
concern an operator that can be described as a generalized circular operator with a given 
variance matrix. It's not entirely obvious that the operator T studied in j3] and in the 
present article is actually of this form. A proof is supplied in Appendix IA1 below. 

Theorem 2.5. Thm. 5] Let be the trace preserving conditional expectation ofW*(D , T) 
onto V = W*(D ), which we identify with L°°[0, 1] as in [7\. Let k G N and let (Pk,n)ri=o be 
the sequence of polynomials in a real variable x determined by: 

P k fi{x) = 1 (2.6) 
Pk% x ) = Pk,n-i(x + 1), n = 1, 2, . . . (2.7) 
PUO) = Pk,n(°) = ■■■ = ^M 1 ^) = 0, n = 1, 2, . . . (2.8) 
where Pjf^ denotes the £th derivative of Pk, n - Then for all k,n G N, 

E v (((T k )*T k ) n )(x) = P Kn {x), x G [0, 1]. 
Remark 2.6. The above Theorem is equivalent to Thm. 5] because 

E v (((T k )*T k ) n )(x) = E v ((T k (T k )T)(l -x), x G [0, 1]. 
Sniady used Theorem 12 .51 to prove the following formula, which was conjectured in §9]. 
Theorem 2.7. [7, Thm. 7] For all n,k G N: 



tr(((T k YT k r) = . (2.9) 

Sniady proved that Theorem 12 .51 implies Theorem 12. 71 bv a tricky and clever combinatorial 
argument. In the course of proving Theorem 12 .2[ we also obtained a purely analytic proof 
of Thm. I2~31 =^> Thm. 12.71 (see (jH.2j) and Remark l4.Hjl . Note that it follows from Theorem 12.71 
that S% = k k {T k )*T k has the same moments as (T*T) k . Hence the distribution measures 



iik 
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Ps k and Ht*t in Prob(R) are equal. In particular their supports are equal. Hence, by jU 
Thm. 8.9], 

a(S k )=a(T*T) = [0,e}. (2.10) 

We will use Theorem 12.51 to derive in Theorem 12.81 an explicit formula for the measure 
P-DoA defined in f!2.4|) . The formula involves Lambert's W function, which is defined as the 
multivalued inverse function of the function C9zh ze z . We define a function p by 

p(z) = -W (-z), zgC\[±,oc), (2.11) 

where Wq is the principal branch of Lambert's W-function. By §4], p is an analytic 
bijection of C\[i,oo) onto 

Q = {x + iy | —n < y < n, x < y cot y}, 

where we have used the convention OcotO = 1. Moreover, p is the inverse function of the 
function / defined by 

f(w) = we~ w , w G Q. 
Note that / maps the boundary of Q onto [=, oo), because 

f (9 cot 9±i9) = f (-^e^ 6 ) = J- e -^^e (212) 



x sin 9 J sin 9 

and 9 i — ► si|V cote is a bijection of (0, tt) onto (0, e) (see U §8]). By flTUD , it also follows 
that if we define functions p + ,p~ : [-, oo) — > C by 

p ± f JL e ~ ecote j = OcotO ±i9, 0<0<7i, (2.13) 



_ sin 
then 

p ± (x) = limp(x ± iy), xG[^,oo). 
In particular p + Q) = p~ fi) = 1. 

Theorem 2.8. Let fc G N k fixed. Define for t > - and j = 0, . . . , k the functions a,j(t), 
Cj (t) by 

a (t) = p + (t) 

aj (t) = p(texp(i 2 -f)), l<j<k-l (2.14) 
a k (t) = p (t) 

and 

Then the probability measure PD ,s k 071 ^(D ) x a(Sk) = [0, 1] x [0, e] is absolutely continuous 
with respect to the 2- dimensional Lebesgue measure and, with <p as in (|2.1)L /ias density 

for x G (0, 1) and y G (0, e). 

We will prove Theorem 12.21 bv combining Lemma [2.41 and Theorem 12.81 (see Section EJ). 
Finally, we will prove the following characterization of the subspaces "Kt (see Section EJ). 



d ^ S ^ y) = V (y) (^cAv- 1 )^-^ ) (2.16) 



3=0 
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Theorem 2.9. For every t £ [0, 1], 

<K t = {£ £ IK | limsup (-||T fe e|| 2/fc ) < 0- (2-17) 
In particular, "K t is a closed, hyperinvariant subspace ofT. 

3. Proof of Theorem 12.81 for k = 1 

This section is devoted to the proof of Theorem 12.81 in the special case k — 1, which is 
somewhat easier than in the general case. For k — 1 it is easy to solve equations (|2.6|) - (|2.8j) 
explicitly to obtain 

P 1 , n (x) = ±-x(x + n) n -\ (n>l). (3.1) 
n! 

From ()3.1|) one immediately gets ()2.9|) for k — 1, because 



tr((T*T) n ) = / Pi, n (x)dx 
Jo 



.(n + 1)! 

Lemma 3.1. For x £ R and z £ C, Id < -, one /ias 



(x- \){x + n) n 



i 



o (n + 1)!" 



(3.2) 



p{z)x 
n=0 

where p: C\ [i, oo) — > C is the analytic function defined in 



Proof. Note that p(0) = 0,p'(0) = 1. Let p(z) = Y^=ilnZ n be the power series expansion 
of p in B (0, 1). The convergence radius is -, because p is analytic in B (0, 1) and - is a 
singular point for p. Hence for \z\ < 1 and x £ C, the function (z,x) i— > e p ^- ,x has a power 
series expansion 

oo 
£,m=0 

Since 



6 pW 



oo 



m=0 



and since the first non-zero term in the power series for p(z) m is z m , we have cg m = for 
I < m. Hence 

oo 

e^* = Y^Q^Y (3-3) 

£=0 

where Q^(x) is the polynomial Ylm=o c tmX m . Putting z = in ()3.3|) we get QoOe) = 1 and 
putting x = in ()3.3j) we get Q n (0) = for n > 1. Moreover since p{z)e^ p ^ = z for 
C\ [J,oo), we get 

jL( e p(z)*) = p( z ) e PW x = p(z)e- p(z) e p{z){x+1) = ze p{z){x+1) . 
dx 



\z\<K 
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Hence differentiating ()3.3|) . we get 

oo oo oo 

e=o e=o 1=1 

Therefore Q' t (x) = Qt-x(x + 1) for £>1. Together with Q (x) = 1, Q e {x) = 0,{£> 1), this 
proves that Qe(x) = Pi,e(x) for £ > 0. □ 

Remark 3.2. From Lemma 13.11 and (|3.1|) we can find the power series expansion of p(z), 
namely 



p(z) = ze** = £ «,<!)**" = E = E rf" (3-4) 

n=0 n=0 ' n=l ^ 

Similarly one gets 

11 00 i 00 / i \n— 1 i 00 n 

JL = i e -*> = E JV . ( _ 1) ^. = i-E = j - E (3. 5 ) 

rv 7 n=0 n=l n=0 v y 

The latter formula was also found in ^fl §8] by different means. Actually, both formulae can 
be obtained from the Lagrange Inversion Formula, (cf. [HJ Example 5.44]). 

Lemma 3.3. For every x G [0, 1] there is a unique probability measure v x on [0, e] such that 

f y n ■ du x {y) = P ljn (x), n G N . (3.6) 
Jo 

Proof. The uniqueness is clear by Weierstrass' approximation theorem. For existence, recall 
that cr(-D) = [0, 1] and, by [4, §8], a(T*T) = [0,e]. Let now /i = Hd ,t*t denote the joint 
distribution of Do and T*T in the sense of f!2.4j) . For x = 0, v x = 5q (the Dirac measure at 
0) is a solution of (j3.(ij) . Assume now that x > and let e G (0,x). Then for n G No, 

P 1>n {x')dx' = [ l [x _ £tX] (x')P ltn (x')dx' = tr(l [x ^ x] (D)E v ((T*T) n )) 



= tr(l [x _ £tX] (D)(T*T) n ) = J J l [x . £ , x] (x')y n dp(x',y). 

[0,1] x [0,e] 

Let u E)X denote the Borel measure on [0, e] given by v £ ^ x (B) = ^p([x — e, x] x B) for any Borel 
set B in [0, e\. Then by the above calculation, 

/ y n dv £ , x (y) = - f P hn {x')dx'i n G N . (3.7) 

JO £ Jx-e 

Since Pi :Q (x') = 1, is E>x is a probability measure. By (|3.7|) . v £)X converges as e — > in the 
u>*-topology on Prob([0,e]) to a measure v x satisfying (|3.6|) . □ 

Lemma 3.4. Let x G [0, 1]. 

(a) For A G C\[0, e], the Stieltjes transform (or Cauchy transform) of v x is given by 



G X (X) = -exp f p I - ) x ) . (3i 
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(b) Ifx £ (0,l),du x (y) = h x (y)dy, where 

h x (y) = i-Im (exp (V Qj xjj , ye (0,e\. (3.9) 

Proof, (a). Since G X (A) = J Q e j^diy x (y) is analytic in C\[0,e], it is sufficient to check ()3.8|) 
for |A| > e. In this case, we get from Lemma f3. 31 and Lemma f3. II that 

G X (X) = jr-^ fy n dv x {y) = jf2\~ n P n (x) = Uxp (p (±) x) . 

n=0 ^° n=0 V V / / 

(b). For y £ (0, e], put 

h x (y) = — lim ha.(G x (y + iz)) = — — Im (exp ( p~ ( - J x 
ix z^o+ V V \yj 

1 tafexp^n* 



Try V V VI/, 

It is easy to see that the above convergence is uniform for y in compact subsets of (0, e], 
so by the inverse Stieltjes transform, the restriction of v x to (0, e] is absolutely continuous 
with respect to the Lebesgue measure and has density h x (y). It remains to be proved that 
u x ({0}) = 0. But 

( r ui ^ 

Urn XG X (X) = u x {{0}) + ffin J -U—dv x {y) = u x {{0}). 

\(0,e] / 

However, XG X (\) = exp (p (V) x) — > as A — > CT, because x > and lim 2/ _ > _ 00 p(y) = — oo. 
Hence ^({0}) = 0, which completes the proof of (b). □ 

Proof of Theorem, \2. 8i for k = 1. Put /i = Hd ,t*t as defined in ()2.4|) . For m, n £ N we get 
from Lemma [3.31 and Lemma f3.4[ 



y ^ x"V y) = tr(D™(T*T) n ) = lx{D™ E v {{T*T) n )) = £ x m P hn [ 



x)dx 



[0,l]x[0,e] 



x m J^ y n dv x (y)dx = ^ U x m y n h x (y)dyjdx. 

Hence by the Stone- Weierstrass Theorem, p is absolutely continuous with respect to the two 
dimensional Lebesgue measure on [0, 1] x [0, e], and for x £ (0, 1), y £ (0, e), we have 

dfx(x,y) 



h x (y) = ^Im(ex P (V (~J xjj. 



dxdy '''''' -■■ x 1 x ' 1 ■■ 1 11 

We now have to compare (j3.10|) with (j2.16|) in Theorem 12.81 Putting k — 1 in (j2.14j) and 
f!2.15j) one gets for t > \, 

a {t)=p + {t), ai {t)=~p~Hf) 

and 

\p + it)? „ |P + WI 2 



2z Im(p+(t)) ' w 2* Im(p+(t)) ' 
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Hence the RHS of ()2.16j) becomes 



<p(y)c ( ~j (exp (V xj - exp f p+ f - jar 



Im p H 



Im I exp I p 



Substituting now y = ^ e vcotv with < v < vr as in (Q, by and (J2HJ we get 

2 



+ i 



Imp+ i 



sin ve 



-V COt D 



sin 2 v 



1 



Hence f|3.10|) coincides with (|2.16j) for k = 1. 



(3.11) 



□ 



4. A GENERATING FUNCTION FOR SnIADY'S POLYNOMIALS FOR k > 2 



Throughout this section and Section k is a fixed integer, k > 2. 
Lemma 4.1. Let «i, . . . , be distinct complex numbers and put 



: — a,- 



3 



n. 



Then 



( k 



= 1 



ft 



^ lj a p j =0 for p=l,2,...,k-l. 



Proof. We can express ()4.2j) as 



1 


1 . 


1 ■ 




7i~ 




T" 


tt'i 


a: 2 






72 







at 1 




■ a k J 




.7ft. 




.0. 



(4.1) 



(4.2) 



(4.3) 



where the determinant of the coefficient matrix is non-zero (Vandermonde's determinant), so 
we just have to check that (|4.1j) is the unique solution to ()4.3j) . Let A denote the coefficient 
matrix in ()4.3jl . Then the solution to ()4.3)1 is given by 



~7i" 




T 


72 


= A~ 1 





_7fc. 




0. 
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Hence jj = (— l) J+l d ^^' ) , where A\j is the (1, j)th minor of A. By Vandermonde's formula, 

det A = Y\ {a m ~ ae) 



£<m 



and 



Hence 



det (Ay) = («!•■ • a.j-i) (a j+ i ■ ■ ■ a k ) ] [ (a m - ai). 



£<m 



(-1Y+ 1 n at 

e<j e>j ^ 

We prove next a generalization of Lemma f3. II to k > 2. 

Proposition 4.2. Let (Pk,n)™=o be the sequence of polynomials defined Theorem \2. 51 For 
z G C ; |z| < - and j = 1, . . . , k, put 

a J (z)=p(ze^) (4.4) 

I lla ( ( 2 )- Qi W r- 
lAz) = \ W ( 4 -5) 
[ l/k, z = 0. 

Then 

00 k 
n=0 j=l 

/or all z E B (0, ^) and a// s G R. 

Proof. Since p is analytic and one-to-one on C\ [-,oo), it is clear that cnj(z) is analytic in 
B (0, i) and 7y(z) is analytic in B (0, i) \{0}. Using p(0) = and p'(O) = 1, one gets 

iim 7j( ,) = n 7 A = n ( x - ^ (*nr j j • 

1 — exp u — £ — l m= i \ \ / / 

But the numbers exp U^jp) , m = 1, • • • , k — 1 are precisely the k — 1 roots of the polynomial 

S( z ) = Z —^- = z k - 1 + z k ' 2 + + 
z — 1 

Hence 

l 1 5 1 o^ W = 5[l) = l = ^ (0) - 
Thus 7j is analytic in B (0, |) . The RHS of (|4.fij) is equal to 

oo 
£=0 
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where 

k 

Pt(z) = ^2lj{z)k e ocj(zy. 

3=1 

Since Qj(0) = 0, the coefficients to 1, z, . . . , z l ~ x in the power series expansion of (3i{z) are 
equal to 0. Hence 

k oo 

J>;(*)e fcoyW!B = J2 ^rnx e z m (4.7) 

3=1 £,m=0 

where /3^ m = when m < I. But, by the definition of oij(z) and jj(z) the LHS of ()4.7|) is 
invariant under the transformation z — > e l ~k z. Hence /3^ m = unless m is a multiple of fc. 
Therefore 

k oo 

^ 7 ^)e to ' (z > = ^i^(:r)z n * (4.8) 

3=1 n=0 

where 

i?„(x) = ^/5 £ , nfc x £ (4.9) 

1=0 

is a polynomial of degree at most nk. To complete the proof of Proposition 14. 21 we now have 
to prove, that the sequence of polynomials 

Q n (x) = k' nk R n (x), n = 0, 1, 2, . . . (4.10) 

satisfies the same three conditions ()2.6|) - (j2.8|) as P^ n . Putting z = in (|4.8jl we get 

k 

Q Q (x)=R Q {x)=Y,lM = l - 

3=1 

Moreover by ()4.5|) 

\n=0 J j=l 

By definition of p, p(z)e~ p ^ = z for all z e C\ Q, oo). Hence 

(a i (z)e-^«)*=(«e i5 if0* = «* ) j = 1, . . . , fc. 

Thus 



v n=0 / 3=1 n=0 



= fc fc ^ J R n _ 1 (x + l)z nfc 

n=l 

so differentiating termwise, we get 

^(x) = fc fc J R n _ 1 (x + l), n>l 

and thus Qn (x) = Q n -±(x + 1) for all n > 1. We next check the last condition ()2.8|) for the 
<5n, i-e. 

Q n (0) = Q' n (0) = ... = Qt 1] (0) = 0, n > 1. 
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If we put x = in (|4.5jl . we get 

co k 
n=0 j=l 

where the last equality follows from (|4.2|) in Lemma 14.11 Hence Q n {0) = -Rn(O) = for 
n > 1 . For p — 1, . . . ,k — lwe have 



ft 



n=0 \j=l / x=0 3=1 

where we again use (|4.2|) from Lemma 14.11 Hence Qn (0) = k~ nk Rn\o) = for all n = 
0, 1, 2, . . . and p = 1, . . . , k — 1. 

Altogether we have shown that (Q n (x))^L satisfies the defining relations ()2.6|) - (j2.8J) for 
Pk,n{x), and hence Q n (x) = P kn (x) for all n and. This proves (|4.6j) . □ 



Remark 4.3. Based on Proposition ^. ^ we give a new proof of the implication TheoremW. 
=> Theorem "WTi Put 

s k , n = tr(((T k )*T k ) n ) = [ P k , n (x)dx. 

Then by flj~ED 



o 



oo k „x 

Y, s ^(kz) nk = Y,lAk) e ka ^ x dx (4.11) 

n=0 1=1 



for all z G B (0, -\ . By definition, the function p satisfies 

p(s)e-^ = s, seC\[l,oo). 

Therefore, 



aj{z) k e- ka ^ z) = {ze^f = z k 



for allzEB (0, J) . Hence for z E B (0, ±) \{0} ; 



,k aj (z)x dx = _( e *«jW - 1) = _ a .^fc-J 



kaj(z) kz k kctj(z) 

By Lemma \J~7\ we have X^=o 1j( z ) a j( z ) k 1 = 0- H ence by (j4.11j) . 



A; ' a,-m 

n=0 j=l JV ' 

To compute the right hand side of (|4.12|) . we app/y i/ie residue theorem to the rational 
function f(s) = Yli=i 7^1' s e C\{0, ai, «2, • • • , «fc}- following computation z is 

fixed, so let us put ctj = aij(z), jj = Jj(z). Note that f has simple poles at a%, . . . , a k and 
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Moreover f has a second order pole at and Res(f; 0) is the coefficient of s in the power 
series expansion of s 2 f(s) = Y\ e=1 (l — ^.e. 



Res{f- 0) = Vi. 

< * m ■ 

3=1 

Since f(s) = 0(\s\^^ k+2 ^) as \s\ — > oo, we have 



lim y f(s)ds = 0. 

dB(0,R) 

Hence, by the residue Theorem, Res(f; 0) + Yjj=i -^ es (/i a j) = 0> giving 

£% = t<- < 4 - 13 > 

27ms, 6?/ JOZD , we 

ra=0 j=l j=l 

5y ()3.5p . p(2;) _1 — \ — Em=o (tTTi)! ' 2 "' W ^never < |z| < i. i/ence 

j=l fc | m V ' n=0 v ' 

So by comparing the terms in (I4.14|) and (|4.15jl . we </e£ Sfc n = ^Ti)! as desired. □ 

5. Proof of Theorem 12.81 for > 2 

Lemma 5.1. Piti fi fe = {z G C | z k £ [e~ fc ,oo)} and define (Xj(z), Jj(z), j = l,...,k by 
(J4.4|) and (|4.5j) /or a// z G Ofc. 27ien for every x G R, t/ie function 

k 

M x (z) = J2lj(z)e kajiz)x (5-1) 
j'=i 



/.s 



analytic in and for every t G [~, oo), i/ie following two limits exist: 



M+(t) = lim Mj(z), M~(t) = lim M^z). 

z— *t z—*t 
Im 2>0 Im z<0 

Lei and Cj(t) for t > - and j = 0, . . . , k be as in Theorem \2.t\ Then for t > -, 

I-M+(t) = ^|^ Cj (t )e ^^. (5.2) 



INVARIANT SUBSPACES OF THE QUASINILPOTENT DT-OPERATOR 



13 



Proof. Since p: C\ [^,00) — > C is one-to-one and analytic, it is clear, that M x is defined 
and analytic on Vt k . Moreover for t > -, 



and similarly 



Moreover 



lim 0^(2:) 
z— »t 

Im 2>0 



p(te^), j = l,...,fc-l 
p + (t), j = k 

a j(t), j = 1 , . - - , — 1 

ao(*), j = k 



lim (^(z) = dj(t), j — 1, . . . , k. 

Im z<0 



f n 



a t (t)-a,j{t) 



, j = 1, . . . , k - 1 



lim 'Jj(z) = < 

Im z>0 



0<^</c-l 

IT a e (t)-aj(t) 

o<e<k-i 



lim 7 -(z) = | j 



z-t-t 
Im z<0 



ae(t) — dj(t) ' 



j,...,k. 



Hence the two limits M£(t) and Mj. (i) are well defined and by relabeling the kth term to 
be the Oth term in case of M+(t) one gets: 



fc-i 



M A + (t) = £ 

j=0 



\ 



n 



t 0<€<*-l ^ " 



kaj(t)x 



n 

1 i<e<k 



a e (t) 



ag(t) - a,j{t) 



kaj(t)x 



(5.3) 



(5.4) 



/ 



It is clear, that M x (z) = M x (z), z e Q k . Therefore M~(t) = M+(t) and 



ImM+(t) = i(M+(t)-M-(t)). 



Hence for t > -, 



Im M+(t) = J2 b j( t ) ekaj{t)x 
3=0 
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where 



wo =i n 



b (t) = ]_( go(0 a fc(*) ^ TT g^j 

A ' 2i \a (t) - aj (t) a k (t) - a 3 {t)) ^ ^ a e (t) - a (t) 



1+3 

ae(t) 

2t ij<\-i a ^- ak ^y 

Using (j2.15|) and the identity 



m*)=4 n 



a (t) a fc (t) aj(t){a k (t) - a {t)) 



a (t) - ctj(t) a k (t) - a,j(t) (a (t) - a i (t))(a fc (t) - aj(t)) ' 
one observes that for all j G {0, 1, ... , k} 

_ 1 a (t)-a k (t) _ Im p+(t) 
A ) ~ 2i ka (t)a k {t) A) ~ k\p+{t)f A) ' 
This proves (|5.2|) . □ 
We next prove results analogous to Lemma f3. 31 and Lemma f3 .41 for k > 2. 

Lemma 5.2. For every x G [0,1], there is a unique probability measure v x on [0,e fc ], such 
that 

/ u n du x (u) = k nk P k:n (x), ne N . (5.5) 
Jo 

For A G C\[0, e k ], the Cauchy transform of v x is given by 

G X (X) = jJ2^( X ~ lk ) ekajiX ~ h)x ( 5 - 6 ) 
i=i 

where Qf 3 -,7j are azwen fry (14 .4|) and (J4.5j) ana 1 A" 1 ^ zs iae principal value of (-\f\)~ l . More- 
over, the restriction of v x to (0, e fc ] zs absolutely continuous with respect to Lebesgue measure, 
and its density is given by 

j=0 

for u G (0, e k ). 

Proof. By Theorem 12.51 

k nk P k , n {x) = E D (k nk ((T k yT k ) n )(x) = E D (Sf)(x)), x G [0, 1]. 

Moreover cr(S k ) = o-(S k ) k = [0,e k ] by (|2.1(Jj) . Hence the existence and uniqueness of v x can 
be proved exactly as in Lemma 13.31 From Proposition 14. 2[ we get that for |A| > e k , the 
Stieltjes transform G X (X) of v x is given by 

1 00 1 k 1 

G,(A) = -J2\- n k nk PUx) = ^X)7i(A-*)e fa ^"*> 

n=l j=l 
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Let M x (z),z G Qk and M+(t),M~(t), t > 1/e be as in Lemma EHJ Then it is easy to see 
that the function 

M x (z), z <eVL k 
M-(z), ze[l/e,oo) 



M x {z) 

is a continuous function on the set 



-1 

x + iy | x > 0, -Tj— < y < f . 

Hence, by applying the inverse Stieltjes transform, we get that the restriction of v x to (0, e k ] 
is absolutely continuous with respect to the Lebesgue measure with density 



h x (u) = lim lm(G x (u + iv )) = — 

7T l>->0+ ITU 



1 / fe 

- lim Im s T 1 Az)e ka ^ z 

Im z<0 x 

-— Im M-( M - 1/fe ) = — Im M+(u~ 1/fc ). 



7TU TIU 

Hence, by Lemma f5. II we get that for u e (0,e fc ), 

fc 



By dSHD, 



Hence 



_ 1 Im (p+(u V fc )) ^ 1/fc ka ^ u -i /k)x 
nx[U) ~ ttu k\p+(u-V k )\ 2 A ] 



1 Im {p + {l/y)) 
f(v) = 1 tjt) \, , 0<y<e. 



h x (u) = ^M^ ^ Cj (n-^) e ^("- 1/fc )-. [5.1 

j=0 



□ 



Remark 5.3. In order to derive Theorem 12.81 from Lemma 15 .2[ we will have to prove 
^({0}) = for almost all x G [0, 1] w.r.t. Lebesgue measure. This is done in the proof of 
Lemma 15.41 below. Actually it can be proved that ^({0}) = for all x > 0. This can be 
obtained from the formula 

v x ({0}) = I™ XG X (X) 

A->0" 

(cf. proof of Lemma I3.4|) together with the following asymptotic formula for p(z) for large 
values of \z\: 

d c-g(iogM ))' 

log \z\ 

where log(— z) is the principal value of the logarithm. The latter formula can also be obtained 
from 2,, pp. 347-350] using (l2~TTl) . 

Lemma 5.4. Let v = ^d q sI be the measure on [0,1] x [0,e k ] defined in (12 .4j) . Then v is 
absolutely continuous with respect to the Lebesgue measure, and its density is given by 

dv[x, u) 



p(z) = - log(-z) + log(log(-z)) + O 



dxdu 

where h x {u) is given by (|5.8|) . 



h x (u), ze(0,l), «6(0,e fc ), 
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Proof. For m,n G N we have from Lemma 15.21 and Theorem 12.51 that 

x m u n du(x, u) = ti(D™S kn ) = ti(D™E D (S kn )) (5.9) 

[0,l]x[0,e fc ] 

r 1 ( r pk 

jm l ;„nfc 



x m (k nk P k , n (x))dx = J x m yj u n dv x (u)j dx. 

Put g(x) = u x ({0}), x G [0, 1]. From the definition of v x it is clear that x — * v x is a 
w* -continuous function from [0, 1] to Prob([0, e k }), i.e. 



x -> / f(u) dv x (u) } x G [0, 1] 
Jo 

is continuous for all / G C([0, e k ]). Put for j G N, 

fj(u) = 



j, 0<u<l/j 
0, u>l/j. 



Then ^(x) = lim ( f e fAu)dv x {u) ), and hence g is a Borel function on [0, 1]. Putting now 
m = in (|5.9j) we get 



-ikn\ 



ti(Sr) = j \J u n h x (u)du ) dx, n = l,2,... (5.10) 



and for n = we get 



1= / g(x)dx + I I / h x (u)du ) dx. (5-11) 
Jo i o \ Jo 



Let A G Prob([0,e fc ]) be the distribution of S k . Then 

u n d\(u) = tr{S kn ) 



so by ()5.10|) and (|5.11j) . A({0}) = L g(x)dx and A is absolutely continuous on (0, e k ] w.r.t. 
Lebesgue measure, with density u — > J h x (u)dx, u G (0,e fc ). However by ()2.9|) and 
(T*T) k have the same moments. Thus S 1 ^ and (T*T) k have the same distribution measure. 
By (gfl §8]), ker(T*T) = ker(T) = {0}. Hence A({0}) = 0, which implies that g(x) = for 
almost all x G [0, 1]. Thus, using ()5.9|) . we have for all m,n G N 



x m u n dz/(x, u) = j x m | / M n /i x .(M) du ) da;. 

[0,l]x[0,e fc ] 

Hence by Stone-Weierstrass Theorem, v is absolutely continuous w.r.t. two dimensional 
Lebesgue measure, and 

^^- = h x (u), are (0,1), «G(0,e fe ). □ 
ax era 
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Proof of Theorem \2. 8i for k > 2. Let /, g be bounded Borel functions on [0, 1] and [0, e] re- 
spectively, and put 

g 1 (u)=g(u 1 / k ), ue[0,e k }. 

By Lemma f5.4t 



tr(f(D )g(S k )) = tT(f(D ) 9l (S K k )) = J J f(x) gi (u)h x (u)dxdu 

[0,l]x[0,e fc ] 

f(x)g(y)h x (y k )ky k - 1 dxdy 

[0,l]x[0,e] 

where the last equality is obtained by substituting u = y k , y G [0, e]. Hence the measure 
^D ,s k is absolutely continuous with respect to the two dimensional Lebesgue measure, and 
by (|5.8|) the density is given by 

oo 

Ml/*)*!/* -1 = v(y)J2cj^y aii » )X 

3=0 

for x G (0, 1), y G (0,e). □ 

6. Proof of Theorem 12.81 =>■ Theorem 12.21 
Lemma 6.1. Let k G N band let do, . . . , a k be distinct numbers in C\{0} and put 



k 



at — a,- 

1=0 J 

Then 

k 

J>^ = ° P=h2,...,k (6.1) 

j=0 

k 

£> = 1 (6.2) 

3=0 

3=0 j=0 
k 

Z>i a J 2 = E M" 1 - ( 6 - 4 ) 

3=0 0<i<j<k 

Proof By applying Lemma 14.11 to the k + 1 numbers ao, • • • , Ofc, we get (jfi.lj) and ()6.2j) . 
Moreover, ()6.Hj) follows from the residue calculus argument in Remark 14. HI (cf. (|4.1Hjl ). and 
f!6.4|) follows by a similar argument. Indeed, letting g be the rational function 

g(s) = ^TT( ai ), s G C\{0,a ,...,a fc }, 
i=o v 7 
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we have Res(g; a 3 -) = — Yii^j a -a = ~bj a j 2 an d Res^j 0) is the coefficient of s 2 in the 
power series expansion of 

Hence Res(g; 0) = 2^o<j<j<fc( a « a j) _1 - Since g(s) = 0(|s|~( fe+4 )) as |s| — > oo, as in Remark |4~31 
we get 

k 

Res(g; 0) + Res(g; a,) = 0. 

j=0 

This proves ()6.4|) . □ 

Lemma 6.2. Lei &; G N &e /ixerf and /ei aj(t) ; Cj(t) for t G (i, oo) and j = 0, . . . , k be 
defined as in (|2.14j) and (I2.15j) . Ati 



tffot) = ^ Cj {t)e ka ^ t)x , xeK, t>l/e, (6.5) 
j=o 

k 

™(t) = ~Y j a 3 (t)-\ (6.6) 

i=o 

k 



i=o 



Then 



Moreover, if k > 2, then 



and if k > 3, i/jen 



/ H(x,t)dx=l. 
Jo 

/ xH{x 1 t)dx = m(t) 
Jo 



(6i 



(6.9) 



l 

2 



x 2 H(x,t)dx = m(ty + v(t). (6.10) 



Proof. For a fixed t G (-, oo), we will apply Lemma 16.11 to the numbers Oj(t), j = 0, . . . , k 
and 

6i(*) = TT-r^-rr- ( 6 - n ) 

' ^0£(*)-Oi(*) 

Note that by (l2~T5) 

c j (t) = -ka j (t)b j (t). (6.12) 
Since t is fixed, we will drop the t in Oj(t), 6j- (t) and Cj(t) in the rest of this proof. We have 

f 1 H(x,t)dx = ^-(e kaj - 1) = 5^6i(l - e fca 0- (6.13) 
^° <^o ajl i=o 
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Recall that 



a o = P + (t) 



cij = p(te l2 ^ J ), 1 < j < n 



a k = p (t) 

where t G Q,oo). Since p{z)e~ p ^ = z for z G C\ [^,00) we get in the limit z — * t with 
Im z > 0, respectively Im z < 0, that also 

p + (t)e~ p+{t) = p-(t)e~ p ~ (t) =t. 

Hence 



(a 3 e- a ^ k = (te^) k = t k , j = 0, . . . , k, 



which shows 



Mi — ( a 3^ k 



(6.14) 



Hence by lj?TT3|) . (jfi~T|) and (Q we get 

/ iffo t)dx = J2 b J-7kJ2 h 3 a ) = 1 > 

JO ,_n 1 „_n 



i=o 



3=0 



which proves (|6.8|) . Moreover, 

/ xH(x,t)dx = yj(~ fc^j^j 

^0 ■ n 



i=o 



x- 



kdj (kdj)' 



Using (jOlD, jnU and flE3D we get 



J j=0 j=0 j=0 J j=0 3 



m(t) 



provided k > 2. This proves (|6.9p . Similarly 



/ x 2 H(x,t)dx = ^^(—ka,jb 



> 6 J ' C J 



x — 2x 



+ 2- 



fcaj (kcij) 2 (kdj) 3 



3=0 3=0 
Hence by ()6.1|1 and ()6.4j) . we get for > 3 



k k k k 

1 2 2 2 6" 

ii £ «<4 + £ v-ir 1 - m £ + F E i • 



3=0 



1 2 

2 — ' 



a; H(x, t)dx 



y 2 E M^^MEs 71 ) + X>7 2 ] =m(t) 2 + «(t). 
o<i<j<fe y \i=o / 3=0 J 



□ 



The functions H,m,v,a,j,Cj in Lemma 15.21 depend on k G N. Therefore we will in the 
rest of this section rename them Hk,mk,Vk,akj,Ckj- Let F(y) = J Q y <p(u)du, y G [0, e] as in 
Proposition 12.11 Since ip is the density of a probability measure on [0, e], we have 

0<F(y)<l, |/G[0,e]. (6.15) 
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e / 







Lemma 6.3. For t G f~,oo) ; 



Figure 1. The contour C e . 



lim rrikii) — F 

k— »oo V t 

lim i>fc(t) = 0. 



Proof. 



m k (t) 



3=0 \j=0 

where / : [0, 1] — > C is the continuous function 

r P +(t)-\ u = o 

f(u) = < p(te i2nu )-\ < u < 1 

[ p-(t)-\ u = l. 

Hence 



lim rrik(t) 



f(u)du 



1 1 



2n J Q p(te 



w 



-de 



i 

2tH 



dB(0,t) 



zp(z 



-dz. 



(6.16) 
(6.17) 



(6.18) 



To evaluate the RHS of ()6.18|) we apply the residue theorem to compute the integral of 
(zp(z))~ l along the closed path C E , < e < |, which is drawn in Figure [TJ 
Since p(z) ^ when z ^ we have 



dz 



Res 



-;0 



2m } Ce zp{z) \zp{z) 
and by ()3.5|) . Res (-^y , 0J = — 1. Thus, taking the limit e — > + , we get 

\ 



1 

2vn 



/ 



dt r dz r 1/e dt 

J <e + 7 ^pjz) + it tp~{t) 

dB(0,t) 



-1. 
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Since p~(t) = p + {t), we get by (I3~TT1) 

1 f dz i 1 f* 1 / 1 \ , i 1 /■* Imp+(s) , 
' 1 / -Im — — tfa = -1 + - / ; , \' ds 



2ni J zp(z) Wl/e s \P + ( S )J 7T 7 1/e s|p+(V |L ' 

3B(0,t) 

= -1 + / -4<p f - ) = -1 + / <p(u)du 

Jl/eS 2 \Sj J l/t 

= -l + F(l)-F(l/t) = -F(l/t). 
Hence (Ifi.lfiJ) follows from (|fj,18jl . In the same way we get 

Hence 

\2 



lim kvk(t) = / /(«) du, 
fc^oo y 

so in particular 

lim v k (t) = 0. □ 
Proof of Theorem \2. 6 A By Lemma 12.44 Theorem 12.81 and (jfi.5j) . 

||D -F(5 fc )||I= ^ |x-F(y)|V(2/)^fe(x,i)rfxdy. 

[0,l]x[0,e] 

Moreover by f)6.8j) — f)6. 1U() we have for ?/ G (0, e) and > 3, 

\x - F(y)) 2 H k (x, ±)dx = (v k C-) + m fc (I) 2 ) - 2m k Q)F{y) + F(yf 

= (m k C-)-F(y)y + v k (l). 

Hence for k > 3 

II A) - F(S k )\\l = jf ((m fc (i) - F(y)) 2 + v k ($)<p{y)dy. 

Since (p(y)H k (x, i) is a continuous density function for the probability measure /iD s fc on 
(0, 1) x (0, e), and since <p(y) > 0, < y < e, we have H k (x,t) > for all x G (0, 1) and 
* G (i, oo). Thus by (|Qjl - (|07Tjl . m fc (t) and v fc (t) are the mean and variance of a probability 
measure on (0, 1). In particular < m k (t) < 1 and < v k (t) < 1 for all t > 1/e. Hence by 
(jnilEJ), (|6.17|) and Lebesgue's dominated convergence theorem 

lim \\D o -F(S k )\\l = 0. 

fc^oo 

Hence D G W*(T). For < t < 1, the subspace 0~C t = l[o,t](A))^ is clearly T-invariant, 
and since D G W*(T), "K t is affiliated with W*(T). □ 
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7. HYPERINVARIANT SUBSPACES FOR T 

In this section, we prove Theorem 12.91 The proof relies on the following four results. 
Lemma [7.21 is probably well known, but we include a proof for convenience. 

Lemma 7.1. For every fceN, ||T fc || = (|) fc ^ 2 . 

Proof. By (jHUJ), ||T fc || 2 = \\(T*) k T k \\ = k- k \\S k \\ = (f) fc . □ 

Lemma 7.2. Let (5a) AeA be a bounded net of selfadjoint operators on a Hilbert space ft 
which converges in strong operator topology to the selfadjoint operator S G B(ft) ; and let 
o- p (S) denote the set of eigenvalues of S. Then for all t G H\a p (S), we have 

liml ( _ oot] (S , A ) = l(-oo.t](5'), (7.1) 
AeA 

where the limit is in strong operator topology. 

Proof. There is a compact interval [a, b] such that cr(S\) C [a, b] for all A and c(S) C [a, b}. 
Therefore, given a continuous function : R — > R, approximating by polynomials we get 

hm0(5 A ) = 0(5), 
AeA 

in strong operator topology. Let t G R, let e > and choose a continuous function (f> : R — ► R 
such that < < 1, </>(x) = 1 for x <t — e and = for x > t. Then for every £ G IK 

(i(-oo, t - e] o%o < (</>(£)£,£> = iim(0(^)e,e) < iiminf(i ( _ 0Oit] (5' A )e,e). 

AeA AeA 

Hence taking the limit as e — * + , we get 

(l ( -oo,t)(S)£,0 < liminf(l ( _ 00 , t] (^ A )e,e). (7.2) 
AeA 

Similarly, by using a continuous function -0 : R — > R satisfying ip(x) = 1 for x < t and 

0(x) = for x > t + e, we get 

(lC-oo,*] (£)£,£> > limsup(l ( _ OO) i ] (^ A )e,0. (7.3) 
AeA 

If t ^ o- p (S), then r^oo^S 1 ) = l(_ 00)f ](S'), and thus by (|7.2|) and (|7.3|) . we have 

liml ( _ 00it] (S , A) = l(_ 00it] (S'), (7.4) 
AeA J 

with convergence in weak operator topology. However, the weak and strong operator topolo- 
gies coincide on the set of projections in 3 (ft). Hence we have convergence (|7.1|) in strong 
operator topology, as desired. □ 



Proposition 7.3. Let F : [0,e] — > [0, 1] be the increasing function defined in Proposition^ 
and fix t G [0, 1]. Let 

£ 4 = {£ G ft | 3£ fc G ft, lim ||& - e|| = 0, limsup(f \\T%\\ 2 ' k ) < t}. 
T/ien L t = ft F ( et ) . 

Proof. For t = 1, we have by Lemma 17.11 that £-! = ft = fti = 'Kpuy Assume now 
< t < 1, and let f G ft F(et) = l [0)J F (et)] (A))ft = l M (F(D ))0C. Since o- p (D ) = and 
since F is one-to-one, we also have a p (F(D )) = 0. Hence, by Theorem 12 . 81 and Lemma 17^1 

hm i M (5,)e = i M TO)x = e- 

k— »oo 



INVARIANT SUBSPACES OF THE QUASINILPOTENT DT-OPERATOR 23 

Let £& = l[o,et]('Sfc)£- Then as we just showed, lim^oo ||£ — £ fc || = 0. Moreover, since 
(T*) k T k = k lk Sl, we have 

\\T%\\ 2 = k- k (S k k t h ,Z k ) < k~ k {et) k U k f < CjfUW 2 - 

Hence limsup A .^ 00 (|||T fc ^ fc || 2/ ' A: ) < t, which proves "Hpui) Q &t- To prove the reverse inclu- 
sion, let £ G L t and choose £ fc G IK such that 

lim - £11 = 0, limsup (-\\T%\\ 2/k ) < t (7.5) 
By ()2.10p . cr(iSfc) = [0, e]. Let E k be the spectral measure of S k and let 

7jfe (fl) = 

for every Borel set B C [0, e]. Then 7^ is a finite Borel measure on [0, e] of total mass 
7fc([0, e]) = ||£fe|| 2 and for all bounded Borel functions / : [0, e] — > C, we have 

(f(S k )£ k ,£ k )= f fd lk . (7.6) 
Jo 

In particular, 

f e x k d lk (x). 



JO 

Let < e < 1 - t. By IfTSjl . there exists k G N such that f ||T*£ & || 2/fe < t + f for all fc > k . 
Thus, 

x*d 7fc 0r) = (S&*,&> = ^||T fc £ fe || 2 < (e(t + f)) fc , (A; > A*). 



Since ( e( - t ^_^ ) fc > 1 for x G [e(i + e), e], we have 







\ k / , 1 e \ k 



-,([d/- + 0. ( j)< / (^^J d7*(*) < [j-f ) IIUIP 



Hence, by (17. 



)(5'fc)£fc|| = (l(e(t+e),oo)(5'fc)£fc,£fc) < ( J ||£fc 



lr 2 lit II 2 



which tends to zero as k — ■> 00. Since ||£fc — £|| — > as — > 00, we get 

lim ||l( e( t + e),oo)(^fc)£|| = 0, 

k— »oo 

which is equivalent to 

lim l [0 , e ( t+e )](,Sfc)£ = £. 

fc— >oo 

Hence, by Theorem 12.81 and Lemma f7. 21 

l[0,.F(e(t+e))](A))£ = l[ , e ( t+e )](F(D ))£ = £, 

i.e. £ G IKi?(e(t+e)) for all e G (0, 1 — t). Since 

se(F(ei),l) 

it follows that L t C IKp^), which completes the proof of the proposition. □ 
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Lemma 7.4. Let t G (0, 1) and define (a n )^_ 1 recursively by 

a x = F(et) (7.7) 

a n+1 = a n F( — ). (7.8) 



JTien (an,)^! a strictly decreasing sequence in [0, 1] and lim n _>oo a n = t. 

Proof. The function x t— > F(ex) is a strictly increasing, continuous bijection of [0, 1] onto 
itself. By definition, the restriction of F to (0, e) is differentiable with continuous derivative 

F'(x) =4>(x), xG(0,e), 

where is uniquely determined by 

sinv . \ 1 . . . 
expff cotf) = — smuexpf — vcotv). 

V J IX 

As observed in the proof of |4, Thm. 8.9], the map v i— ► exp(t> cot u) is a strictly decreasing 
bijection from (0,7r) onto (0, e). Moreover, 

d v 

— (sinwexp(— vcotv)) = exp(— vcotv) > 

dv sin v 

for v G (0,7r). Hence is a strictly decreasing function on (0,e), which implies that F is 
strictly convex on [0,e]. Hence 

F(ex) > (1 - x)F(0) + xF(e) = x, x G (0, 1). (7.9) 

With t G (0, 1) and with (a n )£° =1 defined by (Z2j) and l|73ty . from ((131) we have «i = *Xet) e 
(t, 1). If a G (t, 1) and if a' = aF(f ), then clearly a' < a. Moreover, by (I7~T?1) . 



(//•' | — 1 > a-- = t. 

a 



Hence (a n )™ =1 is a strictly decreasing sequence in (t, 1) and therefore converges. Let a c 
lim^^oo a n . Then by the continuity of F on [0, e], we have 

/ eb 



a 

Hence F{-^-) = 1, which implies a^ — t. □ 

Proof of Theorem \2.yi Let T = UT(X, A) be constructed using [31 §4], as described in the 
introduction. For t G [0, 1], let 



X t = {£<E'K\ limsup ( -\\T k Z\\ 2/k ) < t}. 

n— >oo \ 6 / 



(7.10) 



We will show 

MtC^C JCF (et) , t g [0, 1]. (7.11) 

The second inclusion in (|7.11|> follows immediately from Proposition 17.31 The first inclusion 
is trivial for t = 0, so we can assume t > 0. Letting P t = l[ 0jt ](_D ) be the projection onto 
!Kf, from [H Lemma 4.10] we have 

Aei ^T\^ t = P t TP t = U7{^ 



T t =—T\ Kt = P t TP t = U7(—P t XP t , A t ), (7.12) 
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where X t : L°°[0, 1] — > P t L(F 2 )Pt is the injective, normal *-homomorphism given by Xt{f) 
X(f t ), where 



Ms) 



f(s/t) ifse[o,t] 

if se (*,!]■ 



Therefore, T t is itself a DT(<5 , l)-operator in (P t A4P t ,t 1 T~\ PtMPt ). Hence, by Lemma I7~T1 
applied to T t , we have, for all £ G "Kt, 

k/2 



\t*Z\\ = < (j;) Ml 



Therefore, limsup fc ^ 00 (f \\T k ^ k ) < t and £ G X t . This completes the proof of (fTTTl) . 

From (|7.11|) . we have in particular 3C = "Kq = {0} and DCi = Jii = "K. Let t G (0, 1) and 
let (a n )^ =1 be the sequence defined by Lemma 17.41 We will prove by induction on n that 
3Q C ?{ an . By (J7TTJ), C JC ai . Let n G N and assume % t C IK an . Then 

3C t = {£ G ?{ an | limsup f-||T fc ef /fc l < *} (7.13) 
= {£ G :K a „ | limsup f-||T a ^|| 2 / fe ) < -}. (7.14) 

But the space (I7.14j) is the analogue of %t/a n for the operator T an . By (I7.11j) applied to the 
operator T an , we have that % t is contained in the analogue of c KF{et/a n ) 

forT an . Using (HQ) 

(with a n instead of t), we see that this latter space is 

^an0-{0,F{et/a n )])5C an = A(l[ , a „,F(et/a n )])^a n = A(l[ ,a n+1 ])^a„ = ^a n+1 ■ 

Thus % t C !K an+1 and the induction argument is complete. 

Now applying Lemma [731 we get ^ DnLi ^ = as desired. □ 

Appendix A. £>-Gaussianity of T, T* 

The operator T was defined in j3] as the limit in *-moments of upper triangular Gaussian 
random matrices, and it was shown in [3] that T can be constructed as T = UT(X, A) 
in a von Neumann algebra Ai equipped with a normal, faithful, tracial state r, from a 
semicircular element IeM with t(X) = and t(X 2 ) = 1 and an injective, unital, normal 
*-homomorphism A : L°°[0, 1] — > M. such that {X} and A(L°°[0, 1]) are free with respect 
to r and t o A(/) = J Q f(t)dt. (See the description in the introduction and jH §4].) Let 
V = A(L°°[0, 1]) and let Ed : M. — > V be the r-preserving conditional expectation onto V. 

In jTj , it was asserted that T is a generalized circular element with respect to Ed and with 
a particular variance. It is the purpose of this appendix to provide a proof. 

Lemma A.l. Let f G L°°[0, 1]. Then 

E v (TX(f)T*) = X(g), (A.l) 

Ed{T*X{J)T) = A(/i), (A.2) 

E v (TX(f)T) = 0, (A.3) 

E v (T*X(f)T*) = 0, (A.4) 
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where 

•1 



g(x) = / f(t)dt, h(x) = / f(t)dt. (A.5) 

Moreover, 

E V (T) = 0. (A.6) 
Proof. From [J| §4], lim n ^. o0 ||T — T„|| =0, where 

2™-l 

i=i 

and p[a, 6] = A(l[ 0) 6]). Therefore, 

lim \\E v (T\(f)T*) - E v {T n \{f)T* n )\\ = 0. 

n^oo 

We have 

2 n -l 

E v {T n \{f)T* n ) = T£p[L±,±} Ev (Xp[i,l}\(f)X). 

3=1 

Fixing n and letting a = Jj/ 2 n f(t)dt, we have 

Xp[&, l]X(f)X = 1]A(/) - a)X + a(X 2 - 1) + a, 

and from this we see that Er>(Xp[£;, l]X(f)X) is the constant J\, 2 „ f(t)dt. Therefore, we 
get E v (T n X(f)T*) = \(g n ), where " 

7/ /2B /(t)df if ^ < x < ±, j e {l, . . . , 2 n - 1} 



jf*L=k< x <i. 



2" 

Letting n — ► oo, we obtain (|A.1|) with g as in (|A.5|) . 

Equations (|A.2|) - (|A.4|) and ()A.6|) are obtained similarly. □ 

Comparing Sniady's definition of a generalized circular element (with respect to T>) in [7] 
with Speicher's algorithm for passing from "D-cummulants to "D-moments in [HJ §2.1 and 
§3.2], we see that an operator S G L(F 2 ) is generalized circular if and only if all T>- 
cummulants of order fc ^ 2 for the pair (S,S*) vanish. Hence S is generalized circular 
if and only if the pair (S, S*) is D-Gaussian in the sense of [HI Def. 4.2.3]. Thus, in order to 
prove that T has the properties used in [Zj, it suffices to prove the following. 

Proposition A. 2. The distribution of the pair T, T* with respect to Ed is a T>-Gaussian 
distribution with covariance matrix determined by (|A.lJ) - (|A.6j) . 

Proof. Take Xi, X 2 , . . . € A4, each a (0, l)-semicircular element such that 

v, ({^}); =1 

is a free family of sets of random variables. Then the family 

of *-subalgebras of M. is free (over V) with respect to Ed- Let Tj = VL7(Xj, A). Then each 
Tj has P-valued ^-distribution (with respect to E%>) the same as T. Therefore, by Speicher's 
P-valued free central limit theorem jHJ Thm. 4.2.4], the T>-valued ^-distribution of Tl+ ^ Tn 
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converges as n — > oo to a "D-Gaussian *-distribution with the correct covariance. However, 
Xl+ j-f Xn is a (0, l)-semicircular element that is free from V, and 

Ti + h T n _ , X 1 + • • • + X n x 

" ( ' )- 

Thus Tl+ '"+ Tn itself has the same D-valued *-distribution as T. □ 
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